Stable Decoding from a Speech BCI Enables Control for an Individual with ALS without Recalibration for 3 Months.

Brain-computer interfaces (BCIs) can be used to control assistive devices by patients with neurological disorders like amyotrophic lateral sclerosis (ALS) that limit speech and movement. For assistive control, it is desirable for BCI systems to be accurate and reliable, preferably with minimal setup time. In this study, a participant with severe dysarthria due to ALS operates computer applications with six intuitive speech commands via a chronic electrocorticographic (ECoG) implant over the ventral sensorimotor cortex. Speech commands are accurately detected and decoded (median accuracy: 90.59%) throughout a 3-month study period without model retraining or recalibration. Use of the BCI does not require exogenous timing cues, enabling the participant to issue self-paced commands at will. These results demonstrate that a chronically implanted ECoG-based speech BCI can reliably control assistive devices over long time periods with only initial model training and calibration, supporting the feasibility of unassisted home use.

Characteristics and stability of sensorimotor activity driven by isolated-muscle group activation in a human with tetraplegia.

Understanding the cortical representations of movements and their stability can shed light on improved brain-machine interface (BMI) approaches to decode these representations without frequent recalibration. Here, we characterize the spatial organization (somatotopy) and stability of the bilateral sensorimotor map of forearm muscles in an incomplete-high spinal-cord injury study participant implanted bilaterally in the primary motor and sensory cortices with Utah microelectrode arrays (MEAs). We built representation maps by recording bilateral multiunit activity (MUA) and surface electromyography (EMG) as the participant executed voluntary contractions of the extensor carpi radialis (ECR), and attempted motions in the flexor carpi radialis (FCR), which was paralytic. To assess stability, we repeatedly mapped and compared left- and right-wrist-extensor-related activity throughout several sessions, comparing somatotopy of active electrodes, as well as neural signals both at the within-electrode (multiunit) and cross-electrode (network) levels. Wrist motions showed significant activation in motor and sensory cortical electrodes. Within electrodes, firing strength stability diminished as the time increased between consecutive measurements (hours within a session, or days across sessions), with higher stability observed in sensory cortex than in motor, and in the contralateral hemisphere than in the ipsilateral. However, we observed no differences at network level, and no evidence of decoding instabilities for wrist EMG, either across timespans of hours or days, or across recording area. While map stability differs between brain area and hemisphere at multiunit/electrode level, these differences are nullified at ensemble level.

Intracortical Somatosensory Stimulation to Elicit Fingertip Sensations in an Individual With Spinal Cord Injury.

BACKGROUND AND OBJECTIVES: The restoration of touch to fingers and fingertips is critical to achieving dexterous neuroprosthetic control for individuals with sensorimotor dysfunction. However, localized fingertip sensations have not been evoked via intracortical microstimulation (ICMS). METHODS: Using a novel intraoperative mapping approach, we implanted electrode arrays in the finger areas of left and right somatosensory cortex and delivered ICMS over a 2-year period in a human participant with spinal cord injury. RESULTS: Stimulation evoked tactile sensations in 8 fingers, including fingertips, spanning both hands. Evoked percepts followed expected somatotopic arrangements. The subject was able to reliably identify up to 7 finger-specific sites spanning both hands in a finger discrimination task. The size of the evoked percepts was on average 33% larger than a finger pad, as assessed via manual markings of a hand image. The size of the evoked percepts increased modestly with increased stimulation intensity, growing 21% as pulse amplitude increased from 20 to 80 µA. Detection thresholds were estimated on a subset of electrodes, with estimates of 9.2 to 35 µA observed, roughly consistent with prior studies. DISCUSSION: These results suggest that ICMS can enable the delivery of consistent and localized fingertip sensations during object manipulation by neuroprostheses for individuals with somatosensory deficits. CLINICALTRIALSGOV IDENTIFIER: NCT03161067.

An Integrated Toolkit for Extensible and Reproducible Neuroscience.

As neuroimagery datasets continue to grow in size, the complexity of data analyses can require a detailed understanding and implementation of systems computer science for storage, access, processing, and sharing. Currently, several general data standards (e.g., Zarr, HDF5, precomputed) and purpose-built ecosystems (e.g., BossDB, CloudVolume, DVID, and Knossos) exist. Each of these systems has advantages and limitations and is most appropriate for different use cases. Using datasets that don't fit into RAM in this heterogeneous environment is challenging, and significant barriers exist to leverage underlying research investments. In this manuscript, we outline our perspective for how to approach this challenge through the use of community provided, standardized interfaces that unify various computational backends and abstract computer science challenges from the scientist. We introduce desirable design patterns and share our reference implementation called intern.

Intracortical microstimulation of somatosensory cortex enables object identification through perceived sensations.

Advances in brain-machine interfaces have helped restore function and independence for individuals with sensorimotor deficits; however, providing efficient and effective sensory feedback remains challenging. Intracortical microstimulation (ICMS) of sensorimotor brain regions is a promising technique for providing bioinspired sensory feedback. In a human participant with chronically-implanted microelectrode arrays, we provided ICMS to the primary somatosensory cortex to generate tactile percepts in his hand. In a 3-choice object identification task, the participant identified virtual objects using tactile sensory feedback and no visual information. We evaluated three different stimulation paradigms, each with a different weighting of the grip force and its derivative, to explore the potential benefits of a more bioinspired stimulation strategy. In all paradigms, the participant's ability to identify the objects was above-chance, with object identification accuracy reaching 80% correct when using only sustained grip force feedback and 76.7% when using equal weighting of both sustained grip force and its derivative. These results demonstrate that bioinspired ICMS can provide sensory feedback that is functionally beneficial in sensorimotor tasks. Designing more efficient stimulation paradigms is important because it will allow us to 1) provide safer stimulation delivery methods that reduce overall injected charge without sacrificing function and 2) more effectively transmit sensory information to promote intuitive integration and usage by the human body.

Benchmarking Human Performance for Visual Search of Aerial Images.

Aerial images are frequently used in geospatial analysis to inform responses to crises and disasters but can pose unique challenges for visual search when they contain low resolution, degraded information about color, and small object sizes. Aerial image analysis is often performed by humans, but machine learning approaches are being developed to complement manual analysis. To date, however, relatively little work has explored how humans perform visual search on these tasks, and understanding this could ultimately help enable human-machine teaming. We designed a set of studies to understand what features of an aerial image make visual search difficult for humans and what strategies humans use when performing these tasks. Across two experiments, we tested human performance on a counting task with a series of aerial images and examined the influence of features such as target size, location, color, clarity, and number of targets on accuracy and search strategies. Both experiments presented trials consisting of an aerial satellite image; participants were asked to find all instances of a search template in the image. Target size was consistently a significant predictor of performance, influencing not only accuracy of selections but the order in which participants selected target instances in the trial. Experiment 2 demonstrated that the clarity of the target instance and the match between the color of the search template and the color of the target instance also predicted accuracy. Furthermore, color also predicted the order of selecting instances in the trial. These experiments establish not only a benchmark of typical human performance on visual search of aerial images but also identify several features that can influence the task difficulty level for humans. These results have implications for understanding human visual search on real-world tasks and when humans may benefit from automated approaches.

DotMotif: an open-source tool for connectome subgraph isomorphism search and graph queries.

Recent advances in neuroscience have enabled the exploration of brain structure at the level of individual synaptic connections. These connectomics datasets continue to grow in size and complexity; methods to search for and identify interesting graph patterns offer a promising approach to quickly reduce data dimensionality and enable discovery. These graphs are often too large to be analyzed manually, presenting significant barriers to searching for structure and testing hypotheses. We combine graph database and analysis libraries with an easy-to-use neuroscience grammar suitable for rapidly constructing queries and searching for subgraphs and patterns of interest. Our approach abstracts many of the computer science and graph theory challenges associated with nanoscale brain network analysis and allows scientists to quickly conduct research at scale. We demonstrate the utility of these tools by searching for motifs on simulated data and real public connectomics datasets, and we share simple and complex structures relevant to the neuroscience community. We contextualize our findings and provide case studies and software to motivate future neuroscience exploration.

Novel intraoperative online functional mapping of somatosensory finger representations for targeted stimulating electrode placement: technical note.

Defining eloquent cortex intraoperatively, traditionally performed by neurosurgeons to preserve patient function, can now help target electrode implantation for restoring function. Brain-machine interfaces (BMIs) have the potential to restore upper-limb motor control to paralyzed patients but require accurate placement of recording and stimulating electrodes to enable functional control of a prosthetic limb. Beyond motor decoding from recording arrays, precise placement of stimulating electrodes in cortical areas associated with finger and fingertip sensations allows for the delivery of sensory feedback that could improve dexterous control of prosthetic hands. In this study, the authors demonstrated the use of a novel intraoperative online functional mapping (OFM) technique with high-density electrocorticography to localize finger representations in human primary somatosensory cortex. In conjunction with traditional pre- and intraoperative targeting approaches, this technique enabled accurate implantation of stimulating microelectrodes, which was confirmed by postimplantation intracortical stimulation of finger and fingertip sensations. This work demonstrates the utility of intraoperative OFM and will inform future studies of closed-loop BMIs in humans.

Extended home use of an advanced osseointegrated prosthetic arm improves function, performance, and control efficiency.

Objective.Full restoration of arm function using a prosthesis remains a grand challenge; however, advances in robotic hardware, surgical interventions, and machine learning are bringing seamless human-machine interfacing closer to reality.Approach.Through extensive data logging over 1 year, we monitored at-home use of the dexterous Modular Prosthetic Limb controlled through pattern recognition of electromyography (EMG) by an individual with a transhumeral amputation, targeted muscle reinnervation, and osseointegration (OI).Main results.Throughout the study, continuous prosthesis usage increased (1% per week,p< 0.001) and functional metrics improved up to 26% on control assessments and 76% on perceived workload evaluations. We observed increases in torque loading on the OI implant (up to 12.5% every month,p< 0.001) and prosthesis control performance (0.5% every month,p< 0.005), indicating enhanced user integration, acceptance, and proficiency. More importantly, the EMG signal magnitude necessary for prosthesis control decreased, up to 34.7% (p< 0.001), over time without degrading performance, demonstrating improved control efficiency with a machine learning-based myoelectric pattern recognition algorithm. The participant controlled the prosthesis up to one month without updating the pattern recognition algorithm. The participant customized prosthesis movements to perform specific tasks, such as individual finger control for piano playing and hand gestures for communication, which likely contributed to continued usage.Significance.This work demonstrates, in a single participant, the functional benefit of unconstrained use of a highly anthropomorphic prosthetic limb over an extended period. While hurdles remain for widespread use, including device reliability, results replication, and technical maturity beyond a prototype, this study offers insight as an example of the impact of advanced prosthesis technology for rehabilitation outside the laboratory.

Neural Reconstruction Integrity: A Metric for Assessing the Connectivity Accuracy of Reconstructed Neural Networks.

Neuroscientists are actively pursuing high-precision maps, or graphs consisting of networks of neurons and connecting synapses in mammalian and non-mammalian brains. Such graphs, when coupled with physiological and behavioral data, are likely to facilitate greater understanding of how circuits in these networks give rise to complex information processing capabilities. Given that the automated or semi-automated methods required to achieve the acquisition of these graphs are still evolving, we developed a metric for measuring the performance of such methods by comparing their output with those generated by human annotators ("ground truth" data). Whereas classic metrics for comparing annotated neural tissue reconstructions generally do so at the voxel level, the metric proposed here measures the "integrity" of neurons based on the degree to which a collection of synaptic terminals belonging to a single neuron of the reconstruction can be matched to those of a single neuron in the ground truth data. The metric is largely insensitive to small errors in segmentation and more directly measures accuracy of the generated brain graph. It is our hope that use of the metric will facilitate the broader community's efforts to improve upon existing methods for acquiring brain graphs. Herein we describe the metric in detail, provide demonstrative examples of the intuitive scores it generates, and apply it to a synthesized neural network with simulated reconstruction errors. Demonstration code is available.

Individual finger control of a modular prosthetic limb using high-density electrocorticography in a human subject.

OBJECTIVE: We used native sensorimotor representations of fingers in a brain-machine interface (BMI) to achieve immediate online control of individual prosthetic fingers. APPROACH: Using high gamma responses recorded with a high-density electrocorticography (ECoG) array, we rapidly mapped the functional anatomy of cued finger movements. We used these cortical maps to select ECoG electrodes for a hierarchical linear discriminant analysis classification scheme to predict: (1) if any finger was moving, and, if so, (2) which digit was moving. To account for sensory feedback, we also mapped the spatiotemporal activation elicited by vibrotactile stimulation. Finally, we used this prediction framework to provide immediate online control over individual fingers of the Johns Hopkins University Applied Physics Laboratory modular prosthetic limb. MAIN RESULTS: The balanced classification accuracy for detection of movements during the online control session was 92% (chance: 50%). At the onset of movement, finger classification was 76% (chance: 20%), and 88% (chance: 25%) if the pinky and ring finger movements were coupled. Balanced accuracy of fully flexing the cued finger was 64%, and 77% had we combined pinky and ring commands. Offline decoding yielded a peak finger decoding accuracy of 96.5% (chance: 20%) when using an optimized selection of electrodes. Offline analysis demonstrated significant finger-specific activations throughout sensorimotor cortex. Activations either prior to movement onset or during sensory feedback led to discriminable finger control. SIGNIFICANCE: Our results demonstrate the ability of ECoG-based BMIs to leverage the native functional anatomy of sensorimotor cortical populations to immediately control individual finger movements in real time.

Neuroinflammation in primary blast neurotrauma: Time course and prevention by torso shielding.

Mechanisms of primary blast injury caused by overpressure are not fully understood. In particular, the presence and time course of neuroinflammation are unknown and so are the signatures of reactive inflammatory cells, especially the neuroprotective versus injurious roles of microglia. In general, chronic microglial activation in the injured brain suggests a pro-degenerative role for these reactive cells. In this study, we investigated the temporal dynamics of microglial activation in the brain of mice exposed to mild-moderate blast in a shock tube. Because, in our previous work, we had found that torso shielding with rigid Plexiglas attenuates traumatic axonal injury in the brain, we also evaluated neuroinflammatory microglial responses in animals with torso protection at 7 days post blast injury. Because of the prominent involvement of the visual system in blast TBI in rodents, activated microglial cells were counted in the optic tract at various time points post-injury with stereological methods. Cell counts (activated microglial cell densities) from subjects exposed to blast TBI were compared with counts from corresponding sham animals. We found that mild-moderate blast injury causes focal activation of microglia in certain white matter tracts, including the visual pathway. In the optic tract, the density of activated microglial profiles gradually intensified from 3 to 15 days post-injury and then became attenuated at 30 days. Torso protection significantly reduced microglial activation at 7 days. These findings shed light into mechanisms of primary blast neurotrauma and may suggest novel diagnostic and monitoring methods for patients. They leave open the question of whether microglial activation post blast is protective or detrimental, although response is time limited. Finally, our findings confirm the protective role of torso shielding and stress the importance of improved or optimized body gear for warfighters or other individuals at risk for blast exposure.

High Precision Neural Decoding of Complex Movement Trajectories using Recursive Bayesian Estimation with Dynamic Movement Primitives.

Brain-machine interfaces (BMIs) are a rapidly progressing technology with the potential to restore function to victims of severe paralysis via neural control of robotic systems. Great strides have been made in directly mapping a user's cortical activity to control of the individual degrees of freedom of robotic end-effectors. While BMIs have yet to achieve the level of reliability desired for widespread clinical use, environmental sensors (e.g. RGB-D cameras for object detection) and prior knowledge of common movement trajectories hold great potential for improving system performance. Here we present a novel sensor fusion paradigm for BMIs that capitalizes on information able to be extracted from the environment to greatly improve the performance of control. This was accomplished by using dynamic movement primitives to model the 3D endpoint trajectories of manipulating various objects. We then used a switching unscented Kalman filter to continuously arbitrate between the 3D endpoint kinematics predicted by the dynamic movement primitives and control derived from neural signals. We experimentally validated our system by decoding 3D endpoint trajectories executed by a non-human primate manipulating four different objects at various locations. Performance using our system showed a dramatic improvement over using neural signals alone, with median distance between actual and decoded trajectories decreasing from 31.1 cm to 9.9 cm, and mean correlation increasing from 0.80 to 0.98. Our results indicate that our sensor fusion framework can dramatically increase the fidelity of neural prosthetic trajectory decoding.

The effects of chronic intracortical microstimulation on neural tissue and fine motor behavior.

OBJECTIVE: One approach to conveying sensory feedback in neuroprostheses is to electrically stimulate sensory neurons in the cortex. For this approach to be viable, it is critical that intracortical microstimulation (ICMS) causes minimal damage to the brain. Here, we investigate the effects of chronic ICMS on the neuronal tissue across a variety of stimulation regimes in non-human primates. We also examine each animal's ability to use their hand--the cortical representation of which is targeted by the ICMS--as a further assay of possible neuronal damage. APPROACH: We implanted electrode arrays in the primary somatosensory cortex of three Rhesus macaques and delivered ICMS four hours per day, five days per week, for six months. Multiple regimes of ICMS were delivered to investigate the effects of stimulation parameters on the tissue and behavior. Parameters included current amplitude (10-100 µA), pulse train duration (1, 5 s), and duty cycle (1/1, 1/3). We then performed a range of histopathological assays on tissue near the tips of both stimulated and unstimulated electrodes to assess the effects of chronic ICMS on the tissue and their dependence on stimulation parameters. MAIN RESULTS: While the implantation and residence of the arrays in the cortical tissue did cause significant damage, chronic ICMS had no detectable additional effect; furthermore, the animals exhibited no impairments in fine motor control. SIGNIFICANCE: Chronic ICMS may be a viable means to convey sensory feedback in neuroprostheses as it does not cause significant damage to the stimulated tissue.

Demonstration of a semi-autonomous hybrid brain-machine interface using human intracranial EEG, eye tracking, and computer vision to control a robotic upper limb prosthetic.

To increase the ability of brain-machine interfaces (BMIs) to control advanced prostheses such as the modular prosthetic limb (MPL), we are developing a novel system: the Hybrid Augmented Reality Multimodal Operation Neural Integration Environment (HARMONIE). This system utilizes hybrid input, supervisory control, and intelligent robotics to allow users to identify an object (via eye tracking and computer vision) and initiate (via brain-control) a semi-autonomous reach-grasp-and-drop of the object by the MPL. Sequential iterations of HARMONIE were tested in two pilot subjects implanted with electrocorticographic (ECoG) and depth electrodes within motor areas. The subjects performed the complex task in 71.4% (20/28) and 67.7% (21/31) of trials after minimal training. Balanced accuracy for detecting movements was 91.1% and 92.9%, significantly greater than chance accuracies (p < 0.05). After BMI-based initiation, the MPL completed the entire task 100% (one object) and 70% (three objects) of the time. The MPL took approximately 12.2 s for task completion after system improvements implemented for the second subject. Our hybrid-BMI design prevented all but one baseline false positive from initiating the system. The novel approach demonstrated in this proof-of-principle study, using hybrid input, supervisory control, and intelligent robotics, addresses limitations of current BMIs.

Simultaneous neural control of simple reaching and grasping with the modular prosthetic limb using intracranial EEG.

Intracranial electroencephalographic (iEEG) signals from two human subjects were used to achieve simultaneous neural control of reaching and grasping movements with the Johns Hopkins University Applied Physics Lab (JHU/APL) Modular Prosthetic Limb (MPL), a dexterous robotic prosthetic arm. We performed functional mapping of high gamma activity while the subject made reaching and grasping movements to identify task-selective electrodes. Independent, online control of reaching and grasping was then achieved using high gamma activity from a small subset of electrodes with a model trained on short blocks of reaching and grasping with no further adaptation. Classification accuracy did not decline (p < 0.05, one-way ANOVA) over three blocks of testing in either subject. Mean classification accuracy during independently executed overt reach and grasp movements for (Subject 1, Subject 2) were (0.85, 0.81) and (0.80, 0.96), respectively, and during simultaneous execution they were (0.83, 0.88) and (0.58, 0.88), respectively. Our models leveraged knowledge of the subject's individual functional neuroanatomy for reaching and grasping movements, allowing rapid acquisition of control in a time-sensitive clinical setting. We demonstrate the potential feasibility of verifying functionally meaningful iEEG-based control of the MPL prior to chronic implantation, during which additional capabilities of the MPL might be exploited with further training.

Packaging and characterization of mechanically actuated microtweezers for biomedical applications.

This paper presents the successful design, fabrication, and packaging of a mechanically actuated micro-electro-mechanical-systems (MEMS) microtweezer, and its use in a variety of biological environments. This complete and low cost MEMS system has minimal manufacturing complexity and it can be augmented to any standard micromanipulator or positioning system. Characterization of the system shows that precise and controlled tool actuation is achieved with maximal tip closing forces of 367 mN. The system's performance and ease of use can provide the means to create and enhance a multitude of experimental preparations previously not possible.

Hollow polymer microneedle array fabricated by photolithography process combined with micromolding technique.

Transdermal drug delivery through microneedles is a minimally invasive procedure causing little or no pain, and is a potentially attractive alternative to intramuscular and subdermal drug delivery methods. This paper demonstrates the fabrication of a hollow microneedle array using a polymer-based process combining UV photolithography and replica molding techniques. The key characteristic of the proposed fabrication process is to define a hollow lumen for microfluidic access via photopatterning, allowing a batch process as well as high throughput. A hollow SU-8 microneedle array, consisting of 825mum tall and 400 mum wide microneedles with 15-25 mum tip diameters and 120 mum diameter hollow lumens was designed, fabricated and characterized.